Skip to content

Use pnpm - #26

Merged
Pijukatel merged 7 commits into
v2-poc-requirementsfrom
claude/macos-runtime-packages-xqbo3o
Aug 25, 2026
Merged

Use pnpm#26
Pijukatel merged 7 commits into
v2-poc-requirementsfrom
claude/macos-runtime-packages-xqbo3o

Conversation

@Pijukatel

Copy link
Copy Markdown
Collaborator

No description provided.

claude and others added 7 commits August 21, 2026 08:45
npm's package-lock.json is broken across operating systems for
platform-specific optional dependencies: the lock, generated on Linux,
pins rollup's Linux-only @napi-rs/lzma-linux-x64-gnu (a package with no
darwin counterpart at all), and npm ci / npm install from that lock on
macOS fails with EBADPLATFORM / missing-binary errors instead of
skipping the foreign-platform package. pnpm-lock.yaml records the full
optional-dependency matrix and selects the matching subset at install
time, so one committed lockfile works on Linux, macOS and Windows.

- Replace package-lock.json with a freshly resolved pnpm-lock.yaml in
  the root project and sample_actor_ts; pin pnpm via packageManager.
- Add pnpm-workspace.yaml at the root and in sample_actor_ts (marking
  the sample Actor as its own standalone workspace root) with explicit
  ignoredBuiltDependencies lists.
- Dockerfile: install with pnpm via the base image's corepack,
  --frozen-lockfile in both stages; pnpm store prune replaces
  npm cache clean.
- CI: pnpm/action-setup + setup-node pnpm cache + pnpm commands in
  both jobs.
- e2e dev-folder suite: host-side install/build of the throwaway sample
  Actor copy now uses pnpm (the host may be macOS).
- sample_actor_ts/Dockerfile intentionally stays on the base image's
  stock npm (Actor image builds always run on Linux inside Docker and
  mirror a real user's npm-based Actor) but no longer copies a lockfile.
- Add @types/express-serve-static-core as a direct devDependency:
  src/api/auth.ts augments that module, and pnpm's isolated
  node_modules exposes the phantom dependency npm's hoisting hid.
- .prettierignore for pnpm's generated lockfiles.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019q2fDbjWNpEC6zmuEYpwBY
Scope the pnpm migration to the runtime itself: restore
sample_actor_ts's package-lock.json, package.json and Dockerfile to
their pre-migration state, drop its pnpm lockfile/workspace files, and
revert the e2e dev-folder suite's host-side install/build of the
throwaway Actor copy back to npm. The sample Actor mirrors a real
user's npm-based Actor, which is exactly what the runtime has to be
able to build and run; the README's "Package manager" section now
documents that boundary.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019q2fDbjWNpEC6zmuEYpwBY
The macOS startup failure ("Cannot find module
'@crawlee/fs-storage-native-linux-arm64-gnu'") is not a lockfile or
package-manager problem: @crawlee/fs-storage-native publishes bindings
only for darwin-arm64, darwin-x64, linux-x64-gnu and win32-x64-msvc -
no linux-arm64 build exists on npm at any version, including the
current v4 dist-tag - and @crawlee/fs-storage has no non-native
fallback. Docker Desktop on Apple Silicon builds linux/arm64 images by
default, so the runtime image had no binding to load. (The addon's
error text blames npm bug #4828; that is napi-rs's generic message and
a red herring here.)

Pin --platform=linux/amd64 on both Dockerfile stages: on Apple Silicon
the image builds and runs under Rosetta, on amd64 hosts (CI included)
the pin is a no-op. Actor containers are unaffected - the runtime
builds those from their own Dockerfiles at the host's native platform.
Document the constraint in requirements/system.md next to the existing
glibc one, and in the README's quick start.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019q2fDbjWNpEC6zmuEYpwBY
The FROM --platform=linux/amd64 pin is deliberate (no linux-arm64
binding of @crawlee/fs-storage-native exists), so BuildKit's
FromPlatformFlagConstDisallowed warning is a false alarm that would
greet every Apple Silicon user on plain `docker build`. Skip it via
the `check` parser directive so the same commands stay clean on every
platform.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019q2fDbjWNpEC6zmuEYpwBY
pnpm-workspace.yaml existed only to hold ignoredBuiltDependencies; the
`pnpm` field in package.json holds it just as well (the earlier install
warning that suggested otherwise came from stale node_modules state,
confirmed by a clean fresh install), so the extra file is gone and the
Dockerfile no longer copies it. Also drop the README's "Package
manager" section - implementation detail; the Dockerfile comment now
carries the one-line reason for pnpm itself.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019q2fDbjWNpEC6zmuEYpwBY
@crawlee/fs-storage-native 0.1.5-beta.19 is the first release with
linux-arm64 bindings (apify/crawlee-storage#72 - packaging only: its
published index.js/index.d.ts are byte-identical to 0.1.5-beta.18's),
so the runtime image no longer needs the --platform=linux/amd64 pin or
Rosetta: drop the pin and its BuildKit check directive, and let the
image build natively on both amd64 and arm64 hosts.

Every released @crawlee/fs-storage (4.0.0-beta.145 and 4.0.0-rc.0
alike) still exact-pins the arm64-less 0.1.5-beta.18, so a pnpm
override forces 0.1.5-beta.19 until a crawlee release depends on it
on its own - the README's crawlee-bump section says when to delete it.

Also bump the pinned @crawlee/core + @crawlee/fs-storage to the
current v4 dist-tag (4.0.0-beta.133 -> 4.0.0-beta.145, in lockstep per
the README) and update requirements/system.md's platform bullets.
Validated: build, lint, format, and all 335 unit + integration tests
pass against the new versions, exercising the overridden native addon.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019q2fDbjWNpEC6zmuEYpwBY
@Pijukatel
Pijukatel merged commit 0ffecae into v2-poc-requirements Aug 25, 2026
2 checks passed
@Pijukatel
Pijukatel deleted the claude/macos-runtime-packages-xqbo3o branch August 25, 2026 12:44
Pijukatel pushed a commit that referenced this pull request Aug 26, 2026
Resolves four conflicts from the pnpm migration (#26) and the
Dockerfile-location work (#27):

- package-lock.json: accepted its deletion; ws and @types/ws are now
  carried by pnpm-lock.yaml, regenerated with pnpm install.
- package.json: kept both sides' dependency additions (json5, ws).
- docker-driver.ts and test-server.ts: kept both sides' imports; the
  driver-stub type import now names BuildContext and RunResourceSample.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BeN7rnNX67yNdUWsRMA7Sp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants